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SUMMARY 


The  objective  of  this  research  is  to  test  the  feasibility  of  design¬ 
ing  a  close-coupled,  two-waj,  communication  link  between  man  and  computer 
using  biological  information.  The  research  plan  is  to  conduct  experiments 
to  determine  whether  biological  information  from  the  central  nervous 
system  and  muscles  of  portions  of  the  vocal  apparatus  can  be  directly 
related  to  thought  processes;  whether  this  information  can  be  recognized 
by  a  computer;  and  whether  the  biological  processes  (representing  a  parti¬ 
cular  thought)  can  be  induced  in  a  human  user.  Should  such  a  close¬ 
coupling  between  man  and  machine  prove  to  be  feasible,  an  individual 
using  such  a  biocybernetic  communication  system  would  be  able  to  "talk” 
(i.e.,  .both  send  and  receive)  with  a  computer  at  the  speed  of  thought, 
rather,  than  be  limited  by  the  speed  of  a  teletype  or  other  electromechan¬ 
ical  ’device  through  which  ideas  in  the  form  of  questions  and  answers  must 
normally  pass. 

The  research  plan  was  predicated  on  existing  evidence  that  verbal 
ideas  or  thinking  are  subvocally  represented  in  the  facial  muscles  of  the 
vocal  apparatus  (see  Introduction,  p.  4  ,  for  details).  If  the  patterns 
of  this  muscle  activity  are  at  all  similar  to  those  involved  in  normal 
overt  speech,  then  it  is  reasonable  to  assume  that  the  electrical  activity 
of  the  brain  during  covert  speech  (verbal  thinking)  may  be  similar  to 
that  during  overt  speech.  The  objective  of  the  first  year  of  research 
was  to  establish  the  validity  of  this  basic  premise. 

The  general  methodology  was  to  record  the  electromyograph  (EMG)  of 
facial  muscles  involved  in  speech  from  volunteer  human  subjects  during 
performance  of  language  tasks.  The  electroencephalograph  (EEG)  from 
scalp  electrodes  overlying  areas  of  the  cerebral  cortex  involved  in 
speech  was  recorded  simultaneously  (see  Methods,  p.  7  ,  for  details). 

The  resulting  analog  data  were  then  digitized  for  computer  processing, 
and  several  statistics  that  reveal  patterns  of  cortical  activity  were 
calculated.  These  statistics  were  then  used  in  a  computer  pattern  recog¬ 
nition  program  designed  to  identify  features  in  the  physiological  data 
associated  with  specific  words,  whether  overtly  or  covertly  produced. 

This  report  describes  results  of  computer  analysis  of  EMG  and  EEG 
recordings  from  each  of  three  subjects  during  peiformance  of  a  language 
task  0:1  two  separate  occasions  The  purpose  was  to  determine  whether 
the  computer  could  correctly  classify  15  overtly  spoken  English  words 
based  on  the  EMG  and  EEG  electrophysiological  patterns  alone.  Several 
statistics  were  later  applied  to  the  EMG  and  EEG  responses  that  were 
coincident  with  the  15  word  utterances  (each  repeated  ten  times  at  each 
of  the  two  sessions),  but  only  one  statistic  was  found  useful  for  success¬ 
ful  pattern  recogi ition.  This  was  based  on  calculating  an  average  response 


for  each  electrode  for  the  period  three  seconds  before  and  three  seconds 
after  the  onset  of  vocalization  of  a  word.  Each  of  the  15  average  res¬ 
ponses  per  electrode  (six  electrodes,  or  90  average  responses  for  the  15 
words  per  subject  per  session)  then  served  as  a  template  against  which 
individual  responses  were  compared.  These  comparisons  were  made  by 
calculating  the  RMS  (root-mean-square)  difference  between  a  single  res¬ 
ponse  of  each  electrode  and  the  15  word  templates  for  that  electrode. 

The  individual  electrode  response  was  then  classified  as  the  word  for 
that  template  with  which  the  RMS  difference  was  a  minimum  (see  the 
Appendix  for  details  of  matnematics) . 

The  significant  results  were: 

(1)  Both  EMG  and  EEG  responses,  taken  separately  or  together, 
were  used  to  classify  any  one  or  all  of  the  15  overtly  spoken 
words.  The  percentage  of  correct  classifications  for  all 
electrodes  of  the  three  subjects  for  two  sessions  each  ranged 
from  9%  to  84%. 

(2)  Out  of  900  possible  correct  classifications  across  all  subjects 
and  sessions,  74%  of  the  words  were  correctly  classified  by 
EMG  responses  alone,  63%>  by  EMG  plus  all  EEG  responses,  and 
34%  by  EEG  responses  alone  (see  Table  6,  p.  34  ,  for  details). 
Chi  square  tests  of  significance  showed  that  these  correct 
classifications  could  have  occurred  by  chance  with  a  proba¬ 
bility  of  less  than  1  in  1000. 

(3)  Reliability  within  each  subject  from  one  session  to  the  other 
was  high.  Templates  for  one  session  of  a  given  subject  could 
serve  to  correctly  classify  words  based  on  EMG  and  EEG  res¬ 
ponses  of  the  other  session  nearly  as  well  as  templates  with¬ 
in  a  session.  In  addition,  a  higher  rate  of  correct  classi¬ 
fications  was  made  on  the  second  session  for  all  three  subjects 
than  on  the  first,  indicating  a  learning  or  habituation  effect 
that  lowered  response  variability. 

(4)  When  templates  of  one  subject  were  used  to  classify  words  based 
on  individual  responses  of  another  subject,  the  percentage 

of  correct  classifications  for  EEG  responses  was  no  greater 
than  chance  expectation.  The  percentage  of  correct  classifi¬ 
cations  for  EMG  was  greater  than  chance,  but  not  nearly  so 
good  as  within  subjects.  Thus,  it  appears  that  each  subject's 
biological  patterns  associated  with  speech  are  unique. 

(5)  Six  possible  sources  of  error  in  word  classification  were 
identified,  and  their  relative  contribution  to  decreased 
success  was  evaluated.  It  was  determined  that  if  all  sources 
of  error  could  be  eliminated,  significant  gains  in  correct 
word  classification  using  biological  responses  would  be 
achieved  (perhaps  approaching  90%  or  better). 


We  conclude  that  it  is  feasible  for  a  human  verbally  to  communicate 
overtly  with  a  computer  using  biological  information  alone,  with  a  high 
degree  of  accuracy  and  reliability,  at  least  with  a  small  vocabulary. 
During  the  next  year  attempts  will  be  made  to  eliminate  all  sources  of 
error,  to  determine  the  optimum  locations  for  EEG  recording,  and  to 
communicate  (i.e.,  recognize  signals)  with  the  computer  during  silent 
speech  (verbal  thinking)  using  biological  inf ornyation  alone. 
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INTRODUCTION 


Our  approach  is  predicated  on  previous  research  conducted  by  the 
authors  and  others  in  the  areas  of  psychophysiological  measures  of 
thought,  computer  processing  of  ele?.trophysiological  information,  and 
development  of  computer  pattern  recognition  techniques.  This  research 
is  summarized  below  (see  SRI  Proposal  LSU  71-145,  dated  10  December  1971, 
for  details) . 


Early  work  by  Watson  (1930)  indicated  that  verbal  cognitive  processes 
may  be  represented  in  muscle  activity  of  the  vocal  apparatus  as  subvocal 
speech.  McGuigan  (1970),  reviewing  studies  of  such  covert  oral  behavior 
during  the  silent  performance  of  a  language  task,  concluded  that  covert 
oral  behavior  (as  measured  by  the  electromyograph,  or  EMG)  increases 
significantly  in  amount  and  frequency  of  occurrence,  compared  with  when 
the  subject  is  not  performing  a  silent  language  task.  Thus,  verbal 
ideas,  or  thinking,  although  unquestionably  a  central  nervous  system 
process  (MacNeilage  and  MacNeilage,  1971),  has  some  sort  of  peripheral 
representation  in  the  muscles  of  the  vocal  apparatus. 

If  the  patterns  of  this  muscle  activity  are  at  all  similar  to  those 
involved  in  normal  overt  speech,  then  it  is  reasonable  to  assume  that 
the  electrical  activity  of  the  brain  during  covert  speech,  or  thinking, 
may  be  similar  to  that  during  ov«ik  speech.  That  is,  a  measure  of  the 
scalp-recorded  electroencephalograph  (EEG)  of  a  human  during  verbal 
thinking  should  be  similar  to  the  EEG  of  the  same  individual  when  ex¬ 
pressing  the  same  thoughts  vocally. 

However,  previous  examination  of  the  "raw”  EEG  has  not  revealed  any 
obvious  pattern  related  to  overt  or  covert  speech;  it  may  be  that  only 
patterns  of  EEG  activity  between  various  areas  of  the  brain  at  a  given 
moment  are  related  to  speech.  Several  technical  advances  made  in  recent 
years  have  provided  us  with  some  tools  to  deal  with  this  possibility. 

Most  important  is  the  use  of  computer  techniques  for  frequency  analysis 
of  the  real-time  EEG  and  the  development  of  multivariate  statistical 
procedures  (Donchin  and  Lindsley,  1966;  John  et  al, ,  1964;  ana  Rose 
and  Lindsley,  1965).  These  procedures  allow  comparison  of  specific  com¬ 
ponents  of  EEG  waveforms  that  are  known  to  reflect  different  neuro¬ 
physiological  processes.  In  addition,  certain  statistics,  such  as  auto- 
and  cross-spectral  frequency  analysis  (Walter,  1963;  Walter  and  Adey, 

1965),  the  linear  coherence  function  (Adey,  Kado,  and  Walter,  1967), 
and  the  weighted-average  coherence  (Galbraith,  1967),  may  be  used  to 
determine  the  degree  of  interaction  between  two  different  brain  regions. 
Thus,  with  these  tools,  the  EEG  waveforms  from  several  areas  of  the  brain 
that  are  neurophysiologieally  related  to  speech  may  be  examined  to  deter¬ 
mine  whether  their  patterns  of  interaction  are  similar  during  overt  speech 
and  verbal  thinking. 


A  thorough  visual  analysis  of  the  statistical  results  of  these  EEG 
waveforms  would  be  extremely  complicated  and  time-consuming.  Therefore, 
we  have  turned  to  machine  pattern  recognition  techniques  to  analyze  the 
patterns  of  the  EEG  interrelationships  to  be  found  in  the  average  res¬ 
ponses,  cross-spectra,  and  coherence  functions  related  to  covert  and 
overt  speech.  Most  useful  for  this  feasibility  study  are  techniques 
for  on-line  pattern  recognition  (Hall  et  al. ,  1968).  These  techniques 
allow  the  user  to  process  multivariate  data  by  using  all  reasonably 
conceivable  graphic  plots,  and  manipulate  the  data  further  using  appro¬ 
priate  numeric  procedures  available  in  the  computer  system.  Thus,  for 
our  purposes,  a  set  of  statistics  such  as  the  average  responses,  coher¬ 
ence  functions  of  the  EEG,  the  pattern.*  of  the  EMG  changes  with  overt 
speech,  and  other  measures  may  be  plotted  as  a  function  of  each  other 
for  specific  covert  language  tasks  (i.e.,  thinking). 

The  objective  of  the  first  year  of  this  feasibility  study  was  to 
establish  the  validity  of  the  basic  premise  that  patterns  of  biological 
information  can  be  related  to  language  behavior.  This  has  been  accom¬ 
plished  by: 

(1)  Measurement  of  EMGs  of  the  vocal  apparatus  and  EEGs  overlying 
cerebral  areas  involved  in  speech  during  overt  and  covert 
language  tasks. 

(2)  Computer  processing  and  analysis  of  the  averaged  biological 
activity,  the  cross-  and  auto-spectra,  coherence,  and  weighted- 
average  coherence  of  the  EEG  and  EMG  as  related  to  speech. 

(3)  Application  of  computer  pattern  recognition  techniques  to 
determine  if  the  statistical  patterns  of  biological  activity 
from  the  EMGs  and  EEGs  are  similar  during  overt  and  covert 
speech,  and  to  attempt  to  machine-identif y  silent  language 
performance  with  the  best  pattern  recognition  method. 

During  the  first  year  two  experiments  were  conducted,  designated 
Group  X  and  Group  II.  A  complete  description  of  the  methods  and  results 
for  Group  I  data  and  preliminary  results  for  Group  II  data  were  given 
in  the  First  Semi-Annual  Technical  Progress  Report,  dated  October  1972. 
This  report  describes  details  of  the  computer  analyses  of  Group  II  data. 

To  place  the  Group  II  results  in  perspective,  a  summary  of  the  results 
from  Group  I  follows. 

In  the  first  experiment  (Group  I  data),  EMG  and  EEG  records  were 
obtained  during  performance  of  a  language  task  under  various  conditions 
of  stimulus  presentation,  including:  visual  presentation,  overt  response; 
visual  presentation,  covert  response  (silent  reading);  auditory  presen¬ 
tation,  eyes  open,  overt  response;  and  auditory  presentation,  eyes 
closed,  overt  response.  (The  last  two  conditions  were  chosen  because  the 
EEG  is  characteristically  different  when  the  eyes  are  open  compared  with 
closed.)  The  language  task,  recommended  by  a  psychophysiological  lin¬ 
guist  consultant,  consisted  of  words  and  sentences  most  likely  to  reveal 
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patterns  in  the  EMG  during  speech.  In  the  second  experiment  (Group  II 
data),  similar  records  were  obtained,  but  under  slightly  different 
stimulus  conditions  and  with  20  repetitions  per  subject  for  reliability 
tests.  These  conditions  included  visual  presentation  with  overt  response 
of  five  selected  monosyllabic  words,  and  five  bisyllabic  words,  with 
the  accent  on  the  first  syllable  and  then  on  the  second  (to  compare 
effects  caused  by  ordering  the  emphasis),  for  a  total  of  15  words. 

_  Sl^niflcant  results  of  the  Group  I  data  and  preliminary  analysis 
of  the  Group  II  data  were: 

(1)  EMG  patterns  for  each  word  were  specific  for  that  word. 

(2)  EMG  patterns  for  a  given  word  were  consistent,  showing  less 
within  subject  variability  than  between  subject  variability. 

(3)  Averaged  EMG  patterns  for  a  given  word  spoken  by  a  given 
individual  were  sufficiently  consistent  for  that  averaged 
EMG  to  serve  as  a  template  for  identifying  the  same  word  when 
it  was  imbedded  in  a  sentence. 


(4) 


There  was  some  variability  in  EMG  patterns  for  bisyllabic 
words  between  accent  on  the  first  or  second  syllable,  but  it 
was  sufficiently  small  so  that  either  pattern  could  be  used 

visually  to  identify  the  same  unaccented  word  imbedded  in  a 
sentence. 


(5) 


The  pattern  recognition  analysis  carried  out  on  the  Group  I 
data  distinguished  words  beginning  with  "h"  from  all  other 
words. 


data  thG  re®UltS  fr°m  Gr°UP  1  data  3nd  the  re^lts  from  Group  II 

a  a  before  computer  analysis  showed  that  the  EMG  may  be  used  to  identify 

Diied^th  rr»y  SPOken  W°rdS  °£  3  glVe"  ‘"‘“"dual.  The  results  als“iL 
.  at.  P^erns  of  EEG  activity  associated  with  these  words  may  be 
used  to  identify  the  same  word  when  covertly  produced  (as  in  verbal  thinking) 
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METHODS  FOR  GROUP  II  EXPERIMENT 


Subjects 


nirM  If W6re  three  adult-  right-handed,  human  female  volunteers 
ages  21-41,  heremaft^  designated  B,  C,  and  D.  A  total  of  six  experi¬ 
mental  sessions,  each  of  about  2\  hours  duration,  were  all  earned  out 
U!inHg  tfG„Sa'ne  exPerimental  paradigm.  A  given  session  for  a  given  S 

thus  C5  was6theyftI?thS,S  ^  0nd  her  chronolo£ical  session; 

thus  C5  was  the  fifth  experimental  session  for  subject  C.  Before  con- 

severai  °ppar“tus  debugeinB  sessi°"s  —  —  ^ 


Apparatus 


Electrodes  and  Electrode  Placements 

For  surface  recording  of  the  EMG  from  facial  muscles  involve  in 
speech  production,  Beckman  silver,  silver-chloride  miniature  disk  skin 

elPrtr°HeS  (2-"™ .  “Posed)  were  used.  EEG  scalp  electrodes,  reference 
ectrodes,  and  che  ground  electrode  were  Beckman  silver  silver*  nhi 

“dd  dlf  =kl"  «*>*"*«  <S-„  exposed, .  "ZTt Z^llZTllT 
P  yd  the  skin  under  the  left  mastoid  for  EMG  recordings  and  the 
skm  under  the  right  mastoid  for  EEG  recordings.  All  recordings  were 
monopolar  to  record  absolute  potentials  at  the  recordl^  sUe 

°r.:ted  S‘j",oreas  were  first  cleaned  with  acetone  (alcohol  on  the 

th^  shir  and°^lir°nHdHWUh  RedOIt  electr°de  »y  rubbing  it  into 

D  ®  ’  a;d  followed  by  a  second  cleaning  with  acetone.  A  conductive 

ntt  rh  h  J  electrode  was  then  placed  over  each  recording  area  and  ’ 

session  elect^^  C°Uar  t0  the  underlying  skin.  Following  a  recording 

or  alcoho!  rm°vei'  and  cleaned  with  acetone 

.  FlfJr®  1  shows  the  fc'cial  musculature.  Muscles  involved  in  vocali- 
ation  that  are  surf ace-recordable  are  2,  3,  4,  6,  7  8  9  in  11  ii 

and  16.  Each  of  these  locations  was  tested  during  preliminary ’ exDeri- ’ 
ments.  in  Experiment  1,  combined  sites  13/16  and  7/8  were  found  to 
produce  the  most  reliable  integrated  EMG  patterns  during  overt  speech 
and  so  were  used  for  collecting  Group  II  data.  P 

and  Jasper6  1954)^0^ •  ^  l°/2°  SySt6m  °f  EEG  recording  (Penfield 
Gru  T  :f  Locations  F7,  anterior  C5,  T5  and  T6  were  used  for 

oup  II.  Three  of  these  placements  overlie  arpas  , 

ved  in  speech  (Penfield  and  Roberts,  1959),  as  follows:  F7,  Broca”s°  " 
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. 


■  u->  -.a---.*- 


8.  Mentalis  m. 

9.  Quadratus  labn  inferioris  m. 

Heft) 

10.  Triangularis  m.  (left,  cut) 

11.  Zygomaticus  m.  (left,  cut) 

12.  Quadratus  labii  superiors 

m.  (left,  cut) 

13.  Orbicularis  oris  m. 

14.  Caninus  m.  (left) 


16.  Depressor  septi  nasi 

17.  Nasalis  m.  (left) 

18.  Procerus  m. 

19.  Frontalis  m.  (left) 

20.  Frontalis  m.  (right) 

21.  Orbicularis  oculi  m. 

(left) 

22.  Nasalk  m  (rmhtl 


1.  Orbicularis  oculi  m.  (right) 

2.  Quadratus  labii  superioris  m. 

(right) 

3.  Zygomatic  head  of  quadratus 

labii  superioris  m.  (right) 

4.  Zygomaticus  m.  (right) 

5.  Risorius  m.  (right,  cut) 

6.  Triangularis  m.  (right) 

7.  Quadratus  labii  inferioris  m. 


F  EEG  RECORDING 


speech  area;  anterior  C5,  motor  control  of  vocal  musculature;  and  T5 , 
Wierneki's  area  for  speech  organization  and  comprehension.  In  addition, 
location  T6  on  the  right  (nondominant)  hemisphere,  the  homologue  of  T5 
over  the  dominant  hemisphere,  was  employed  as  a  control. 

After  electrodes  were  attached,  Ss  were  seated  in  a  semi-dark, 
electrically  shielded  booth.  All  electrodes  were  plugged  into  a  junc¬ 
tion  box  leading  to  the  recording  equipment.  Electrode  resistances 
were  checked;  if  any  one  electrode  was  found  to  be  greater  than  5000 
ohms,  it  was  removed,  the  skin  further  cleaned  and  conditioned,  and  the 
electrode  replaced.  When  all  electrodes  checked  correctly,  the  S  was 
instructed  in  the  experimental  procedure,  a  microphone  for  recording 
speech  as  placed  in  front,  of  the  S's  mouth,  and  the  recording  session 
begun. 

Equipment 

Figure  3  shows  a  diagram  of  the  equipment  setup.  Electrodes  from 
the  facial  musculature  were  led  first  to  a  Beckman  Model  9852A  EMG 
integrator  coupler,  with  a  fixed  time  constant  of  0.25  sec,  and  pass 
band  of  20  to  5000  Hz.  For  Group  II,  channels  1  and  2  of  the  Dynograph 
were  used  to  record  the  integrated  EMG.  EEG  electrodes  were  led  to 
Beckman  type  9806A  couplers,  within  the  pass  band  set  to  2  to  22  Hz. 
flat;  channels  3,  4,  5,  and  6  recorded  the  instantaneous  EEG.  Channel  7 
recorded  the  voice  output  of  the  microphone;  channel  8  was  not  used. 
Amplitude  normalization  was  carried  out  by  setting  all  like  channels 
(EMG  or  EEG)  to  the  same  gain. 

All  physiological  signals  were  preamplified  by  Beckman  Model  481B 
preamplifiers,  and  were  then  led  in  parallel  to  Beckman  Model  482A 
power  amplifiers  with  calibrated  zero  suppression  and  to  an  Ampex  SP-300, 
seven-channel,  analog  instrument  tape  recorder.  The  output  of  the 
Beckman  power  amplifiers  drove  ink-writing  galvanometers  on  chart  paper 
moving  at  25  mm/sec.  The  output  on  the  chart  paper  could  be  set  by  a 
switch  to  record  either  the  input  to  the  Ampex  tape  recorder  (i.e. , 
"direct"  recording)  or  the  output  of  the  Ampex;  this  feature  enables  the 
investigator  to  calibrate  and  monitor  the  permanent  tape  recording. 

EMG  and  EEG  recordings  were  on  channels  1  through  6  of  the  Ampex,  using 
frequency  modulation,  at  1-7/8  in. /sec  (pass  band  dc-312  Hz,);  voice  was 
recorded  on  channel  7. 

Group  II  data  filled  one  10-|-in.  Ampex  tape  wjth  analog  data. 

These  data  were  then  sent  through  the  data  analysis  system  using  the 
Linc-8  laboratory  instrument  computer  and  a  CDC-6400  computer  (see 
Data  Analysis  section  below). 
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6  Channels 


Procedure 


Language  Task 

On  the  basis  of  results  from  Group  I  data,  15  words  were  selected 
which  had  the  greatest  likelihood  of  reproducing  reliable  EMG  patterns. 

The  15  words  and  a  sentence  containing  all  the  words,  which  are  shown 
in  Table  1,  consisted  of  five  monosyllabic  and  five  bisyllabic  words. 

The  latter  are  phonetically  balanced  words  used  in  two  groups;  one  group 
has  the  accent  on  the  first  syllable,  and  the  other  group  has  the  accent 
on  the  second  syllable.  These  15  words  were  chosen  to  emphasize  rounded 
lip.i ,  spreading  lips,  bilabials,  and  open  lips  in  the  case  of  the  mono¬ 
syllabic  words,  and  to  assess  the  effect  of  emphasis  (pre-  and  post-) 
of  one  syllable  on  the  other  in  the  case  of  the  bisyllabic  words.  No 
covert  responses  were  obtained  with  Group  II  data. 

Stimulus  Presentation 

Each  of  the  individual  words  and  the  sentence  in  Table  1  was  printed 
on  a  35-mm  slide  (white  on  black  to  reduce  glare)  and  presented  to  the 
S  by  projecting  the  word  (or  sentence)  on  a  rear  projection  screen  about 

3  ft  from  the  S's  eyes.  The  subtended  visual  angle  of  the  stimulus  and 

its  intensity  in  the  semi-darkened  room  were  chosen  to  avoid  squinting, 
glare,  or  eye  strain  and  to  reduce  eye  movements. 

After  installation  in  the  recording  chamber,  the  S  was  instructed 
that  she  was  to  relax  with  eyes  closed  while  the  polygraph  and  tape 
recorder  gains  and  filters  were  adjusted  for  proper  EMG  and  EEG  recordings. 
During  that  period,  the  S  was  to  say  her  name  when  asked  (to  calibrate 
EMG  gains  and  the  voice  channel)  and  to  open  or  close  her  eyes  when  asked 
(to  check  lor  ..lpha  in  the  clost.I-eyes  EEG  and  alpha  blocking,  or  desyn¬ 
chronization,  with  the  eyes  open  and  to  check  for  eye  movement  artifact). 
Following  these  adjustments,  the  S  was  told  that  she  would  be  presented 

with  a  list  of  15  words,  one  at  a  time,  for  ten  full  presentations,  plus 

one  sentence  ct  the  end  of  each  word  list.  The  presentations  would  be 
visual.  (The  S  was  shown  a  test  word  on  the  screen  as  an  example.) 

The  S  was  to  sit  relaxed  with  her  eyes  closed.  On  hearing  the  statement 
ready  from  the  experimenter,  she  was  to  open  her  eyes  and  look  at  the 
screen.  In  2-3  sec ,  a  stimulus  word  would  be  projected  on  the  screen 
for  about  3  sec,  during  which  time  she  was  to  read  the  word  aloud  into 
the  microphone.  When  the  projected  word  was  turned  off,  she  was  to  close 
her  eyes  until  the  next  word  was  presented,  and  wait  until  the  next 
ready"  signal. 

At  the  end  of  the  15  words  (presented  randomly  to  obviate  any  anti¬ 
cipatory  effects  in  the  EMG  and  EEG) ,  the  sentence  was  presented  to  the 
subject.  On  the  signal  ready,  the  S  was  to  open  her  eyes  and  look 
at  the  screen.  When  the  sentence  appeared,  she  was  to  read  it  aloud 
at  her  natural  speed.  The  15  words  plus  a  sentence  were  then  presented 
again;  a  total  of  ten  such  presentations  were  given  each  S  per  session. 
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Table  1 


LANGUAGE  TASK  FOR  GROUP  II  DATA 

Monosyllabic 
TIP 
HIT 
HAD 
PUT 
COOL 

Sentence: 

THE  SHIPWRECKED  SCHOOLBOY  HAD  PUT  A  COOL  COUGHDROP  IN  THE  MOUSETRAP 
AND  AIMED  IT  TO  HIT  AND  TIP  OVER  THE  BLACKBOARD. 
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Each  of  the  three  Ss  was  run  a  second  time  not  less  than  one  week 
nor  more  than  one  month  following  the  first  session.  This  set  of 
measurements  was  recorded  exactly  the  same  as  the  first,  and  was  run  to 
determine  within  S  reliability.  Thus  Group  II  data  are  based  on  six 
electrodes  per  S,  for  three  Ss ,  two  sessions  each,  where  each  session 
consisted  of  ten  repetitions  of  15  words  and  one  sentence.  This 
resulted  in  a  total  of  5400  electrophysiological  responses  (6X3X2 
X  10  X  15)  of  6  sec  each  (this  total  does  not  reflect  analysis  of  the 
sentences,  which  has  not  yet  been  completeT"and  will  be  described  in  a 
later  report). 


Data  Analysis 

Editing  and  Digitizing 

As  noted  in  the  First  Semi-Annual  Technical  Progress  Report,  editing 
and  digitizing  of  the  Group  II  data  for  the  CDC-6400  computer  processing 
was  delayed  while  waiting  for  installation  of  a  Pertec  digital  recorder. 
When  this  was  received  in  early  December  1972,  a  Linc-8  program  written 
for  this  purpose,  was  used  to  edit  and  digitize  the  analog  data. 

A  synchronization  signal  on  channel  7  (voice  channel)  of  the  Ampex 
analog  recorder,  which  preceded  each  stimulus  presentation,  caused  the 
Linc-8  to  begin  sampling  the  six  channels  of  data  and  the  voice  channel 
through  analog-to-digital  converters.  A  total  of  7  sec  of  data  were 
sampled  at  42  samples/sec  for  each  of  the  seven  channels.  The  7th  sec 
of  data  was  collected  for  time  justification  of  the  electrophysiological 
response  as  described  below.  (Tie  data  were  stored  in  memory,  and  on 
command  any  two  of  the  six  data  channels  or  the  voice  channel  could  be 
displayed  on  a  two-channel,  cathode-ray  oscilloscope  driven  by  the  com¬ 
puter.  The  display  itself  consisted  of  6  sec  of  data,  or  256  data  points. 
In  addition,  the  scope  displayed  a  vertical  cursor  that  was  exactly 
centered  to  represent  a  zero  point  for  time  justification  of  the  electro¬ 
physiological  response. 

Figure  4  illustrates  the  seven-channel  dynograph  recording  of  the 
EMG  and  EEG  for  one  presentation  of  the  word  "COOL.  "  Note  the  cursor 
line  at  the  onset  of  vocalization  (channel  7)  which  was  used  to  time- 
justify  all  electrode  responses.  Any  two  of  these  channels  could  be 
displayed  on  the  Line-scope,  as  shown  in  Figure  5.  In  Figure  5A,  an 
EMG  response  is  shown  on  the  top  channel;  the  voice  voltage  and  the 
centered  cursor  are  shown  on  the  second  channel.  Note  that  the  vocal 
onset  is  off-centered  on  the  scope,  indicating  that  this  particular 
response  was  not  time  justified  for  vocalization  to  occur  at  exactly 
3  sec  from  the  onset  of  the  display.  By  use  of  a  second  Linc-8  command, 
all  six  channels  of  data  and  the  voice  channel  could  be  rotated  simul¬ 
taneously  into  and  out  of  memory  with  the  extra  7th  sec  to  place  the 
onset  of  vocalization  at  any  desired  point.  This  feature  was  used  to 
shift  the  data  and  voice  channels  (to  the  left  in  this  case)  so  that  the 
onset  of  vocalization  occurred  at  the  centered,  3  sec  cursor,  as  shown 
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in  Figure  5B.  Thus,  after  time  justification,  3  sec  of  data  were  dis- 
played  before  the  onset  of  vocalization  and  3  sec  after.  On  completion 
of  time  justification,  the  six  channels  of  data  were  stored  on  Line 
tape  on  the  computer,  one  channel  per  block  of  tape.  Then  the  next 

stimulus-word-response  was  sampled  from  the  Ampex  recorder,  the  responses 
were  time  justified,  and  the  data  stored.  responses 

When  responses  to  all  150  word  utterances  and  the  10  sentences  for 

wo rdlnd3  J:Ct"SeSf  °n  h3d  b6en  St°red  °n  Linc  tape  channel  per 

word  and  sentence,  for  a  total  of  990  tape  blocks),  another  Linc-8 

prog. am  was  used  to  transfer  this  data  to  the  Pertec  digital  tape  recor¬ 
der  in  ten  files  of  101  records  per  file  (255  samples  per  block  were 
transferred  rather  than  the  256  collected  because  of  an  error  in  the 
rans  er  program).  The  Pertec  tape  was  then  unpacked  on  the  CDC-6400 
computer,  and  the  data  words  were  reordered  in  sequence  for  da^  processing. 


CDC-6400  Response  Classification  by  Averaging 


As  a  first  approximation  for  machine  pattern  recognition  of  the  six 
electrophysiological  responses  for  each  word  utterance,  it  was  decided 
to  use  a  method  of  averages  to  construct  templates  for  response™!! 
classification.  (The  rationale  and  mathematical  equations  for  this 
ana  all  other  computations  described  below,  are  given  in  the  Ipjendix.) 
electrodpUnit  analysis  for  classification  was  the  individual  6-sec 

CDC-6400  oonnPrSG  C°^espondine  to  a  word.  To  determine  whether  the 
CDC  6400  computer  could  correctly  classify  the  word  by  analysis  of  the 

electrophysiological  response  alone,  the  response  was  compared  with  15 
templates  (one  for  each  of  the  15  words) . 

Physiologic!?  ^  FiSUre  6’  3  giV6n  template  was  the  average  elec^ro- 
physiological ,  6-sec  response  for  the  ten  repetitions  of  the  word  for 

a  given  S  on  each  session.  Each  sample  point  of  the  255  samples  in  a 

ec  epoch  was  added  to  each  corresponding  point  of  the  other  nine 

that°point  "since  tT^  *  ten  ^  the  averaSe  response 

that  point.  Since  there  were  six  electrodes  and  15  words,  there  were 

90  templates  altogether  for  each  S  on  each  session,  or  a  total  of  540 

templates  for  the  three  Ss,  two  sessions  each. 

with  ?glYieCtr^  reSponse  for  a  sinSle  word  utterance 

templates  for  that  electrode,  a  root-mean-square 

“  a  IT  W3S  °?1CUlated  between  single  response  and  the 
template,  as  snown  in  Figure  7.  In  the  top  of  the  figure  the  EMC  13/16 

th_  t  °d?  ^6Sponse  to  TIP  2  <second  utterance  of  TIP)  is  compared  with 
the  template  for  COOL.  That  is,  each  sample  of  the  255  samples  in  the 

The  d  W3S  SUbtracted  from  the  corresponding  sample  in  the  template 
The  difference  for  each  sample  point  (only  a  few  ficticious  values  were 
use  in  Figure  7  for  illustration)  was  squared,  and  the  squared  differences 
were  summed  over  the  255  samples.  Dividing  by  255  and  taking  the  square 
root  provided  the  single  RMS  number  for  that  comparison 
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(a) 


COOL  AVE, 


0  0  0  0  0  -3  -10  -15  -25  -6  -7  -2  0  0  0  DIFFERENCE 

0  0  0  0  0  9  100  225  625  36  49  4  0  0  0  D2 

RMS  =  (lD2/255)1/2  =  2.03 


0000100  -1  523  4  -6  000  DIFFERENCE 

0000100  1  25  49  16  36  000  D2 

RMS  =  (SD2/255)1/2  =  0.60 


FIGURE  7  CLASSIFICATION  OF  TIP  2  AS  TIP  WITH  RESPECT  TO  ANOTHER  TEMPLATE 
(COOL)  USING  RMS  DIFFERENCE 


In  the  bottom  of  Figure  7,  the  same  electrode  response  to  TIP  2  is 
compared  with  the  template  for  the  word  TIP.  Since  TIP  2  was  included 
in  making  the  template  for  TIP  (see  Figure  6),  this  template  was  nec¬ 
essarily  biased  in  favor  of  classifying  TIP  2  response  as  TIP.  There¬ 
fore,  for  this  response  and  all  other  responses  compared  with  their 
own  templates,  the  individual  response  was  first  subtracted  from  the 
template.  The  RMS  difference  was  then  calculated  for  the  individual 
response  against  the  unbiased  average  response  for  the  other  nine  utter¬ 
ances  of  the  word. 

After  calculating  the  RMS  difference  for  a  given  electrode  response 
for  a  given  word  against  all  15  templates,  the  computer  then  classified 
the  response  as  the  word  having  the  smallest  RMS  value. ^  For ^example, 
in  Figure  7,  TIP  2  is  classified  as  "TIP"  rather  than  "COOL,"  since 
the  RMS  difference  with  TIF  AVERAGE  minus  TIP  2  is  less  than  the  RMS 
difference  with  COOL  AVERAGE.  If  this  RMS  difference  remained  the 
smallest  when  TIP  2  was  compared  with  all  15  templates,  then  ultimately 
the  TIP  2  response  was  classified  as  TIP. 

In  this  way,  response  classifications  were  made  for  each  of  the  150 
words  per  S  per  session  per  electrode  (Within  Subjects,  Within  Sessions 
classification).  Two  additional  classifications  were  also  made: 

(1)  Within  Subjects  and  Between  Sessions  (to  assess  within  S  reliability); 
and  (2)  Between  Subjects  (to  assess  individual  differences).  In  the 
first  case,  the  six  electrode  responses  for  each  150  words  of  one  session 
for  a  given  S  were  compared  to  the  15  templates  for  the  other  session 
for  that  S  (e.g. ,  C5  responses  with  C6  templates,  and  C6  responses  with 
C5  templates).  In  the  second  case,  the  six  electrode  responses  for  each 

of  the  150  words  of  a  given  S  on  a  given  session  were  compared  with  the 

15  templates  for  another  S  on  a  given  session  (e.g. ,  C5  responses  with 

B5  templates,  or  D4  responses  with  C6  templates). 

Finally,  in  addition  to  obtaining  classification  of  responses  for 
each  electrode,  it  was  decided  to  pool  both  EMG  electrodes,  all  four 
EEG  electrodes,  and  all  six  electrodes  (EMG  plus  EEG)  to  assess  the 
relative  contribution  of  types  and  number  of  electrode  responses  to  the 
computer  classification. 

Determination  of  Critical  Classification  Period 

The  electrophysiological  recordings  shown  earlier  in  Figure  4  to 
the  word  COOL  illustrate  that  only  about  1  sec  of  the  6-sec  epoch  is 
actually  involved  in  making  the  response.  Since  it  is  possible  that 
computer  classification  of  an  electrophysiological  response  might  be 
based  on  that  portion  of  the  response  following  stimulus  onset,  but 
before  the  onset  of  vocalization,  it  was  decided  to  determine  which 
parts  of  the  6-sec  epoch  contributed  to  the  classification  and  to  what 
extent.  To  do  this,  an  F-ratio  was  calculated  for  each  electrode  for 
the  Within  Words  variance  and  Between  Words  variance  for  each  of  the 
255  data  samples  in  the  6-sec  epoch  (see  Appendix  for  details  of  compu¬ 
tation).  If  a  given  sample  point  was  not  contributing  to  the  computer 
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classification  of  the  word,  then  the  ratio  of  the  two  variances  (i.e., 
the  F-statistic)  should  be  small,  On  the  other  hand,  if  the  Between 
Word  variance  was  significantly  higher  than  the  Within  Wore  variance 
for  a  given  sample  point,  then  it  may  be  assumed  with  some  confidence 
(given  by  statistical  tables  for  the  F-ratio)  that  the  particular  data 
point  was  contributing  to  the  classification. 

Examples  of  the  F-ratio  for  the  255  samples  in  the  6-3ec  epoch  for 
an  EMG  and  an  EEG  electrode  are  shown  in  Figures  8  and  9,  respectively, 
for  Subject  C,  Session  5.  Note  that  in  both  cases  the  F-ratio  remains 
small  and  statistically  insignificant  for  about  the  first  100  samples, 
but  becomes  and  remains  significantly  larger  from  about  sample  101  to 
about  sample  200.  For  the  EEG  (Figure  9),  another  portion  between  about 
sample  205  through  230  barely  reaches  significance.  This  means  that 
for  both  EMG  and  EEG  responses  taken  separately,  only  that  portion  of 
the  6-sec  epoch  between  samples  101  and  200  probably  contributes  signi- 
iN^.ntly  to  classification  of  the  response  (that  portion  of  the  EEG 
from  205  to  230  may  also  contribute,  although  less  significantly). 

F-ratios  were  calculated  for  all  Ss,  sessions,  and  electrodes,  with 
essentially  the  same  results. 

The  finding  that  the  samples  from  101  to  200  of  a  response  are  the 
only  portion  of  the  response  contributing  to  the  classification  suggested 
that  calculating  sums  of  squared  differences  for  RMS  values  across  all 
255  samples  may  produce  undesireable  errors.  Accordingly,  each  electrode 
response  was  further  divided  into  three  subgroups,  and  RMS  values  were 
calculated  and  responses  classified  for  each  subgroup,  as  well  as  for 
the  entire  255  samples.  These  subgroups  were  samples  1-100,  101-200, 
and  201-255.  Thus,  for  each  of  the  six  electrodes  taken  separately,  the 
two  EMG  electrodes  taken  together,  the  four  EEG  electrodes  taken  together, 
and  all  six  electrodes  taken  together,  there  were  four  classification 
epochs  for  a  total  of  36  classifications  per  word  per  S  per  session. 

C DC-6400  Response  Classification  by  Other  Statistics 

As  pointed  out  earlier,  there  is  no  a  priori  reason  for  believing 
that  a  particular  statistic  of  the  EEG  and  EMG  will  ultimately  be  a 
better  classifier  of  the  verbal  response  than  any  other.  Consequently, 
several  other  statistics  were  computed,  RMS  differences  were  obtained 
in  the  same  manner  as  with  the  averaged  responses,  and  individual 
electrode  responses  were  classified.  These  statistics  were  auto-  and 
cross-spectra,  linear  coherence,  and  weighted-average  coherence  (see 
the  Appendix  for  the  meaning  of  these  statistics  and  details  of  their 
computations).  These  statistics  are  presumed  to  be  useful,  since  they 
involve  frequency  analysis  of  the  EEG  and  may  be  used  to  determine  the 
degree  of  interaction  between  two  or  more  different  regions  of  the 
brain.  Such  an  analysis  should  provide  information  about  cortical 
organization,  and  therefore  may  show  patterns  of  organization  that  are 
specific  for  particular  responses  to  the  15  stimulus  words.  Except 
for  the  actual  calculation  of  the  statistic  itself,  all  other  compu¬ 
tations  for  response  classification  were  the  same  as  for  the  averaged 
responses  described  above. 
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Tests  for  Statistical  Significance 

To  show  that  correct  classification  of  single  electrode  responses 
by  the  computer  did  not  occur  by  chance,  a  chi  square  was  calculated 
for  each  electrode  subclassification  for  each  S  and  each  session,  between 
sessions  and  between  subjects.  The  "expected"  frequencies  for  evalu¬ 
ating  the  chi  square  were  based  on  classifications  being  randomly  dis¬ 
tributed  across  words  (see  the  Appendix  for  details  of  computation). 

Control  of  Artifact 

In  a  working  biocybernetic  communication  system,  constraints  on  the 
user  against  eye  or  muscle  movement  may  not  be  effective.  Since  such 
movements  often  produce  artifacts  in  electrophysiological  responses 
(especially  in  the  EEG) ,  it  is  imperative  that  a  system  be  designed 
that  is  not  unduly  affected  by  such  artifacts.  That  is,  pattern  recog¬ 
nition  and  classification  should  be  carried  out  successfully  whether 
artifacts  are  present  or  not.  For  this  reason,  it  was  decided  to  analyze 
Group  II  data  without  removing  those  responses  with  known  artifacts 
present;  that  is,  this  was  a  "worst  case"  analysis. 

Nevertheless,  some  effort  was  devoted  to  controlling  and  identi¬ 
fying  artifacts,  and  to  assessing  their  contribution  to  response  classi¬ 
fication.  This  was  accomplished  by  obtaining  responses  in  the  four 
EEG  electrodes  to  five  words  under  conditions  of  both  visual  and  audi¬ 
tory  stimuli  and  with  and  without  known  eye  movements  and  movement 
artifacts.  The  results  of  these  controlled  studies  are  described  below 
under  "Results,  Group  II  Data,  Sources  of  Error."  Finally,  during 
data  collection  proper,  Ss  were  instructed  to  remain  as  relaxed  as 
possible  during  the  response  period,  with  no  more  eye  or  body  movement 
than  necessary. 
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RESULTS  OF  GROUP  II  DATA 


Classification  by  Averages 

Individual  Responses 

Figure  10  illustrates  the  six  electrode  templates  for  Subject  C, 
Session  5,  for  the  word  COUGHDROP,  with  the  accent  on  the  first  syllable. 
Note  that  for  both  EMG  and  EEG  records,  significant  changes  occur  pri¬ 
marily  over  computer  samples  101  to  200,  thus  beginning  about  0.5  sec 
before  onset  of  vocalization  (arrow  at  3.0  sec). 

Figure  11  shows  the  variability  around  the  averages  of  Figure  10, 
the  upper  trace  of  each  pair  being  the  average  plus  one  standard  devia¬ 
tion,  the  lower  trace  the  average  minus  one  standard  deviation.  Note 
that  the  variability  of  each  electrode  signal  is  relatively  small. 

Similar  plots  were  made  for  all  three  Ss  for  the  two  sessions,  each 
with  essentially  similar  results,  indicating  that  it  should  be  feasible 
to  identify  a  given  spoken  word  by  comparing  the  electrophysiological 
response  to  the  templates. 

As  described  in  the  previous  section,  for  each  S  on  each  session, 
the  individual  electrode  response  for  each  of  the  150  word  utterances’ 
was  compared  with  the  15  templates.  These  comparisons  resulted  in  15 
RMS  values,  from  which  the  CDC-6400  computer  classified  the  response 
with  the  word  for  which  the  RMS  value  v/as  a  minimum.  Tables  of  correct- 
responses  were  then  constructed  for  each  S  and  each  session,  showing 
the  number  of  correct  classifications  for  the  150  words  for  each  elec¬ 
trode  and  each  subclassif ication. 

Table  2  shows  these  results  for  Subject  C,  Session  5,  and  illustrates 
the  method.  Along  the  top  row  are  the  15  words.  The  leftmost  column 
gives  the  channel  number  (corresponding  to  Figure  4)  for  the  six  elec¬ 
trodes,  the  combination  of  both  EMG  electrodes,  the  combination  of  the 
four  EEG  electrodes,  and  the  combination  of  all  six  electrodes.  The 
second  column  on  the  left  gives  the  four  subclassifications  for  each 
electrode  and  combination — that  is,  the  sample  points  of  the  6-sec 
response  over  which  classifications  were  made  according  to  the  findings 
of  the  F-ratios  (see  previous  section).  The  table  proper  then  gives 
the  number  of  responses  correctly  classified  for  each  electrode  and 
each  subgroup.  For  example,  in  Channel  1  (EMG  electrode  13/16)  in  the 
portion  of  the  response  from  sample  1-100,  none  of  the  utterances  of 
TIP  was  correctly  classified,  while  six  out  of  ten  were  correctly 
classified  for  samples  iul-200,  none  for  samples  201-255,  and  five  for 
the  entire  response  (samples  1-255).  The  same  interpretation  ^Tbe 
made  for  all  other  words,  electrodes,  and  combinations. 
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FREQUENCY  OF  CORRECT  RESPONSE  CLASSIFICATION,  SUBJECT  C5 
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On  the  far  right,  under  the  column  labeled  "Correct  "  the 
number  of  responses  acrn«?^  4-v,Q  -i  c  ,  , ,  *  tbe  total 

ly  for  subgroup  1-100,  86  out  of ^S^forsubgroupUl-ZOO33^ C°rreCt" 
for  subgroup  201-255,  and  83  out  of  150  for  the  S„tai  ’  5° 

5S.2S  f:v“:iecrdes  ■- 

sample  period^e^ves^fdecrease^h1’631'011863  °UtSide  °f  the  101~200 
more,  all  electrodes  taken  togetheAser^All  "°d  ^laS®if ication-  Further- 
Table  2),  especially  in  t£e  loilonn  it  data  3t  the  bottom  of 

fier  than  any  single  electrode  or  eoph'grtUP’  Serve  as  a  better  classi- 
sample  subgroup,  !o5  oit  !f1s0  Lr  70rroi°n-  ^  ^  th#  101-20° 
classified.  Table  2  also  shows  that  for  COrrectly 

responses  (individually  or  taken  toL^  \  ®  session,  the  EMC 

any  (or  all)  of  the  EEC  responses,  and  that *t he  EE^of  the331^"- 
hemisphere  (T6,  Channel  6)  was  the  poorest  classfner  of  aU 

The  two  columns  on  the  rivht  nf  To o  • 
for  Within  Words  (CHI  1=1  degree  of  frP  h  \  ^ive  the  chi  square  values 
15=15  degrees  of  freedom)  f  *  eedOTn)  and  Between  Words  (CHI 

or  correct  Cla2i£ UToL 

Correct"  ^  ‘U*, 

with  a  probability  of  le*«  ,  freater  than  30  occurred  by  chance 

ci-  ’hoes  Sire's  2  0°11 z  E*rlnation  of  this 

hlBh  levels  of  significance  '* 

subgroup  101-200,  the  best"claMif ier^f t™i)fshoblSth  (?A11  eloctrodes' " 

the  words  that  were  correct! v  r.!  ■  4,  ■  ,  a11'  shows  the  frequency  of 

three  words  (COOL,  BLACKBOARD  and^lOUSETRAP) ^UbdSCt  C’  Session  5.  Thus, 
nine  times  out  of  ten  whil^five  were  correctly  classified 

of  the  time,  two  70%, 'four  50%!  ^0^  IoT  Cl3SSified  8°% 

Within  Subjects 

all  three'sJ  were^u^o^two  °f  "easureme”t5  and  ™sponse  classification, 
than  one  week  IZ  no  loL^r  tb!  each'  The  sessl<">s  »ere  no  less 

ziiiTzr  ~  :-hsaapareachTasb^:ioi  sr 

Sessions!  S  /«‘- 

Subject"?  l~Z  tTJs^  “?  Bet”en  SeSS1°"S  comParisons  for 
combinations  (Tables  4  and  5  giv  er  subgroup  (101-200)  for  all  electrode 
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Table  3 


i' 

h 

NEAREST  PERCENTAGE  CORRECT  RESPONSES  AND  RANK  ORDER  OF  ELECTRODES 
AS  CLASSIFIERS,  SAMPLE  POINTS  101-200,  SUBJECT  C,  WITHIN  AND  BETWEEN  SESSIONS 


Channel 

Within 

Sessions 

Between 

Sessions 

C5  with 

C5* 

C6  with 

C6 

C5  with 

C6** 

C6  with 

C5 

%  Correct 

Rank 

%  Correct 

Rank 

%  Correct 

Rank 

%  Correct 

Rank 

1 

57 

3 

76 

2 

39 

3 

53 

3 

2 

43 

7 

56 

4 

33 

4 

40 

4 

3 

45 

6 

48 

5 

22 

7 

15 

9 

4 

47 

5 

46 

7 

31 

6 

29 

6 

5 

39 

8 

36 

9 

14 

9 

28 

7 

6 

26 

9 

37 

8 

21 

8 

25 

8 

EMG 

63 

2 

79 

1 

42 

2 

64 

2 

EEG 

49 

4 

47 

6 

31 

5 

31 

5 

All 

70 

1 

75 

3 

50 

1 

65 

1 

*  C5  data  compared  with  C5  templates. 
**  C5  data  compared  with  C6  templates. 


30 


....  --■< .Mr^LaBWjaMaUft 


Table  4 


NEAREST  PERCENTAGE  CORRECT  RESPONSES  AND  RANK  ORDER  OF  ELECTRODES 
AS  CLASSIFIERS,  SAMPLE  POINTS  101-200,  SUBJECT  B,  WITHIN  AND  BETWEEN  SESSIONS 


Channel 

Within 

Sessions 

Between 

Sessions 

B5  with 

B5* 

B6  with 

B6 

B5  with 

E',6** 

B6  with 

B5 

%  Correct 

Rank 

%  Correct 

Rank 

%  Correct 

Rank 

%  Correct 

Rank 

1 

49 

3 

57 

4 

32 

4 

52 

3 

2 

59 

2 

59 

3 

56 

3 

51 

4 

3 

9 

9 

22 

8 

19 

9 

23 

9 

4 

29 

7 

14 

9 

23 

8 

24 

8 

5 

31 

6 

27 

7 

25 

6 

37 

5 

6 

38 

5 

29 

5 

29 

5 

30 

C 

EMG 

67 

1 

74 

1 

61 

2 

67 

1 

EEG 

12 

8 

29 

6 

24 

7 

28 

7 

All 

47 

4 

67 

2 

63 

1 

65 

2 

*  B5  data  compared  with  B5  templates. 

**  B5  data  compared  with  B6  templates. 
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Table  5 


NEAREST  PERCENTAGE  CORRECT  RESPONSES  AND  RANK  ORDER  OF  ELECTRODES 
AS  CLASSIFIERS,  SAMPLE  POINTS  101-200,  SUBJECT  D,  WITHIN  AND  BETWEEN  SESSIONS 


Within 

Sessions 

Between 

Sessions 

D3  with 

D3* 

D4  with 

D4 

D3  with 

D4** 

D4  with 

D3 

Channel 

%  Correct 

Rank 

%  Correct 

Rank 

%  Correct 

Rank 

%  Correct 

Rank 

1 

66 

3 

72 

2 

56 

3 

56 

4 

2 

67 

2 

71 

3 

59 

2 

65 

2 

3 

19 

7.5 

27 

8 

11 

8.5 

11 

8.5 

4 

27 

5 

30 

7 

12 

7 

11 

8 .5 

5 

16 

9 

33 

6 

15 

6 

16 

7 

G 

19 

7.5 

19 

9 

24 

5 

17 

5.5 

EMG 

74 

1 

84 

1 

67 

1 

71 

1 

EEG 

25 

6 

37 

5 

11 

8.5 

17 

5.  5 

All 

65 

4 

70 

4 

50 

4 

57 

3 

*  D3  data  compared  with  D3  templates. 

**  D3  data  compared  with  D4  templates. 
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order  of  electrodes  as  classifiers  are  given  by  classifying  each 
response  first  with  the  templates  of  its  own  session,  and  then  with  the 
templates  from  the  other  session.  The  following  may  be  concluded  from 
inspection  of  Tables  3,  4,  and  5: 

(1)  In  all  three  Ss,  a  greater  number  of  responses  were  correctly 
classified  on  the  second  session  (Within  Sessions  comparison) 
than  on  the  first  (by  rank  order  and  not  necessarily  for  a 
particular  electrode  or  combination).  Thus,  an  habituation 
or  learning  effect  is  present  that,  presumably,  reduces  the 
variability  from  one  session  to  the  next. 

(2)  Again  in  all  three  Ss ,  when  the  second  session  responses  are 
compared  with  the  first  session  templates  (Between  Sessions 
comparisons),  a  greater  number  of  responses  were  classified 
correctly  than  when  the  first  session  responses  are  compared 
with  the  second  session  templates.  This  is  also  probably 
due  to  a  decreased  variability  in  the  individual  responses 
of  the  second  session,  further  strengthening  the  conclusion 
that  habituation  or  learning  operates  to  improve  performance 
in  response  classification. 

(3)  In  all  three  Ss,  the  EMG  responses,  taken  separately  or 
together,  are  better  classifiers  than  any  EEG  response  or  the 
EEG  responses  taken  together.  However,  there  appears  to  be 
no  consistency,  either  within  a  S  or  across  Ss,  for  any 
particular  EEG  response  being  a  better  classifier  than  any 
other. 

(4)  The  range  of  percentages  of  correct  classifications  across 
Ss  is  from  9%  to  84%,  with  all  EMG  and  all  electrodes 
generally  in  the  higher  ranges,  EMGs  separately  and  all  EEGs 
in  the  middle  range,  and  EEGs  separately  in  the  lower  ranges. 

(5)  Comparisons  among  percentages  of  correct  responses  among  the 
three  Ss  indicate  that  responses  from  Subject  C  are  better 
classifiers,  in  general,  than  either  B's  or  D's,  with  B 
slightly  better  than  D,  Examination  of  either  raw  data  or 
templates  shows  that  this  result  is  probably  due  to  varia¬ 
bility  (larger  in  D,  less  in  B,  still  less  in  C). 

Tables  6  and  7  give  summaries  for  all  Ss  and  sessions  together  for 
Within  Subjects/Within  Sessions  comparisons,  and  Within  Subjects/ 

Between  Sessions  comparisons,  respectively.  These  tables  are  to  be 
interpreted  as  is  Table  2.  They  show  that  for  all  Ss  and  sessions, 
the  results  are  essentially  the  same  as  for  Subject  C,  Session  5.  That 
is,  the  number  of  correctly  classified  responses  (out  of  a  possible 
900)  is  74%  for  EMGs,  63%  for  all  electrodes,  and  about  34%  for  EEGs. 
Again,  this  is  true  only  for  subgroups  101-200  and  1-255,  the  former 
being  the  better  classifier  of  the  two.  All  chi  squares  for  these  two 
subgroups  (last  two  columns  on  the  right  for  1  and  15  degrees  of  freedom, 
respectively)  were  significant  at  p  v/ell  below  .001. 
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ev  Th®  c°mparison  of  Between  Session  (Table  7)  with  Within  Sessions 
Uabie  6)  for  all  Ss  and  sessions  repeats  the  findings  shown  in  Tables 
3,  4,  and  5— namely,  that  Within  Sessions  responses  are  correctly 
classified  with  a  higher  frequency  than  the  Between  Sessions  responses. 

isTgood  th6Se  flVe  tabl6S  alSO  Show  that  intrasubject  reliability 

Between  Subjects 

shnu,  “UltS  ^  thS  E?G  analysis  from  Group  1  data  (see  Introduction) 
showed  that  an  individual  s  electrophysiological  responses  accompanying 

speech  are  unique  to  that  individual.  Therefore,  a  comparison  was 
made  Between  Subjects  for  one  session  for  all  combinations.  The  results 
of  this  comparison  are  shown  in  Tables  8  and  9.  Table  8  elves  the 
number  of  correct  classifications  for  all  subgroups,  and  their  asso- 

Tnd  ele?tlcTar6l’  wbil*  Table  9  compares  percentage  correct  responses 
and  electrode  rank  order  for  all  Ss  over  all  sessions.  Table  8  shows 

that  a  significant  number  of  EMG  responses  were  correctly  classified 
above  chance  level. 

at  D<T0?letLSl!OW^  t!!a!  GVen  th°Ugh  the  EMG  responses  are  significant 
~J P'*:  tbe  h^hest  frequency  (both  EMG)  is  only  39%,  compared  with 
74%  Within  Subjects/Within  Sessions,  and  62%  Within  Subjects/  Between 

Hhr  ref ltS  SUE8:eSt  that  muscles  act  more  Hke 

each  other  between  Ss  than  homologous  brain  sites.  Table  9  also  shows 

EMGS  takGn  sepa-tely  or  together  are  bet Lr  d^ssifi^s 
an  the  EEGs  taken  separately  or  together,  no  EEG  electrode  is  pre¬ 
ferred  for  classification  purposes.  P 


Sources  of  Error 

As  indicated  in  Data  Analysis  section  of  Methods,  no  special 
ttempts  were  made  to  remove  artifacts  or  other  known  sources  of  error 
because  we  wanted  to  see  how  well  the  computer  would  perform  in  classi- 
ying  responses  under  the  "worst  case"  condition.  Several  known  sources 
of  error  existed,  including  time  and  amplitude  variations  and  muscle 
and  eye  movement  artifacts.  Each  of  these  is  demonstrated  below  and  an 
evaluation  made  as  to  their  relative  contribution. 

Confusion  Matrices  (Bisyllabic  Confusion) 

fused^i^rdhlrtf7!^086  reSponses  not  erectly  classified  were  con- 
for  earh  i  +  ,  mp  ates  >  a  set  of  confusion  matrices  was  constructed 

for  each  electrode,  for  each  S,  and  for  each  session.  Table  10  shows 

such  a  confusion  matrix  for  Subject  C,  Session  5,  for  the  "All  Channels" 
category,  for  each  of  the  four  sample  subgroups.  The  confusion  matrix 
shows  how  each  electrode  response  was  classified.  Note  that  for  sub- 

TlZllf  T  and  201-255  (left  Side  °f  table)-  the  rates  correct 
This  !  arS  qUltS  l0W  and  appear  to  be  randomly  distributed. 

in  contrast  with  subgroups  101-200  and  1-255  (right  side  of  table), 
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Table  9 


NEAREST  PERCENTAGE  CORRECT  RESPONSES  AND  RANK  ORDER  OF  ELECTRODES 
AS  CLASSIFIERS,  SAMPLE  POINTS  101-200,  ALL  SUBJECTS,  ALL  SESSIONS 


Within 

Within 

Sessions/ 

Subjects 

Between  Sessions/ 
Within  Subjects 

Between 

Subjects 

Channel 

%  Correct  Rank 

%  Correct 

Rank 

%  Correct 

Rank 

1 

63 

3 

48 

4 

27 

2 

2 

59 

4 

51 

3 

24 

3 

3 

28 

8 

17 

0 

8 

6.5 

4 

35 

5 

22 

7.5 

7 

8.5 

5 

30 

7 

22 

7.5 

7 

8.5 

6 

28 

9 

24 

5 

8 

6.5 

EMG 

74 

1 

62 

1 

39 

1 

EEG 

33 

6 

24 

6 

9 

5 

All 

65 

2 

58 

2 

22 

4 
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CONFUSION  MATRIX,  SUBJECT  C,  ALL  CHANNELS 


where  a  strong  diagonal  begins  at  the  upper  left  each  cell  nf  ^ 
diagonal  representing  the  number  of  correct  responses  ^ 

The  diagonal  is  seen  even  more  clearly  in  Tables  11  anri  19  u ■  , 
present  the  confusion  matrices  for  all  ^  with*  V*  d  12 '  whlch 
Sessions,  respectivelv  ?  Wlthln  sessi°ns  and  Between 

subgroups  101-200  and  1-255  TheV  ?  two.str°ng  subdiagonals  in 
(column)  and  SRI fl  ' \  G  flrSt  begins  at  the  Junction  of  SB2 

and  SB2  (row).  These' 'subdiagonals  b6glnS  ^  jUnction  of  SB1  (column) 

zzzzmz  1 usr by  the 

suggesting  that  the  effect  of  emphasis  on  class! I  ,  scents, 

This  result  further  suggests  thnt  the  flcatlon  3  s  not  large. 

cations  can  be  materially  increased  if  emphasififnornken^'nto^ 

b™Prse™“‘  ;S'  ^  -"fusions  „f  emphasis 

the  number  of  total  correct  responses'ln^he*  01  r®Sponses-  For  example, 
group  101-200  in  T  hle«  n  ^  ?°  m  the  mam  diagonals  for  sub- 

respectively^  By  di,“  ‘rdi«  L”™  ?92.",d  827'  *  6» 

syllabic  words  il  cuLK^  g  resp„^e"sUSJhese°r  “Ph“SlS  °n  the  M' 

aad  5B4,  or  70,  a„d  03,  correctly-pla^f ZZlliZZ 

confusion  matrices  in  Tables  11  l2  a“d13  rf,  “J"  "  °‘  ^  ^ 

Sion  of  one  word  with  another  oi  one  C°nfU- 

Time  Shift  and  Amplitude  Variation 

fir.t^na^t^n!r,toe?I?f8ePPOr  iUuStrated  in  figure  12.  The 
left)  of  the  electrophysiologic^l^Lponse^urres'^t'to'ihe  (t°  T% 

tz  ZV  ”s 

^r“:„i,rrpsr^  r” ^"-sr 

to  -- zizz 
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SUMMARY  CONFUSION  MATRIX,  ALL  SUBJECTS,  ALL  CHANNELS,  WITHIN  SESSIONS 
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Artifacts 


By  definition,  artifacts  are:  (1)  potentials  causing  a  distur¬ 
bance  not  methodically  related  to  t:  e  original  biological  response  being 
measured;  or  (2)  potentials  that  arc  methodically  related  to  the  res¬ 
ponse  from  a  source  other  than  that  which  is  assumed  to  produce  the 
response.  The  former  disturbances  are  random  and  cannot  contribute  to 
computer  recognition  of  the  response,  although  they  can  degrade  the 
recognition  as  random  noise.  The  latter  are  true  consistent  distur¬ 
bances  and  imply  that  a  particular  assumption  about  the  physiological 
origin  of  the  response  is  incorrect.  However,  from  a  practical  point 
of  view,  if  the  computer  can  recognize  the  biological  response  produced 
y  any  source,  including  one  whose  origin  may  be  mistaken,  then  bio- 
cybernetic  communication  may  still  function  adequately  even  though  all 

the  sources  of  the  response  cannot  be  identified.  Several  such  artifacts 
are  discussed  below. 

Muscle  artifacts  were  controlled  by  having  Ss  relax  as  much  as 
possible  throughout  the  experimental  trials,  with  the  allowance  to  move 
around  between  trials.  Examination  of  the  temporal  changes  of  EMG 
records  in  comparison  with  any  muscle  artifacts  in  the  EEG  revealed 
little  contamination  from  this  source. 

Figure  13  illustrates  a  relatively  rare  occurrence  of  EMG  artifact 
in  which  an  EMG  response  (electrode  7/8)  to  TIP  1  was  classified  as 
MOUSETRAP  The  artifact  itself  is  due  to  a  larger  voltage  Espouse 
than  usual  for  the  amplifier  gain,  resulting  in  saturation.  The  response 
was  misclassif ied  simply  because  the  minimum  RMS  difference  was  obtained 
y  chance,  with  MOUSETRAP,  even  though  the  signal  value  more  closely 

resembles  the  form  of  the  TIP  template.  The  confusion  has  no  other 
significance. 


One  of  the  more  prevalent  artifacts  in  EEG  data  is  the  corneal- 
re  inal-occipital  potential  due  to  eye  movements.  In  reading  the  stimu¬ 
lus  words  used  in  this  project,  such  eye  movements  are  practically 
impossible  to  eliminate  completely.  Nevertheless,  their  contribution 
may  be  assessed  in  various  ways.  For  example,  in  the  collection  of 
Group  I  data,  two  stimulus  inputs  (visual  and  auditory)  were  used  In 
the  visual  input,  both  overt  and  covert  responses  were  obtained  to 

Durw  It  C°rribUti0n  °f  EMG  artifacts  caused  by  speech  (overt  response) 
During  the  auditory  input,  responses  were  obtained  during  eyes  open  and 
eyes  closed  conditions  to  assess  both  the  contribution  of  visually 
evoked  responses  and  eye  movements,  both  presumably  absent  under  these 
conditions.  Results  showed  that  under  all  four  conditions,  certain 
potential  changes  were  present  in  the  EEG  that  could  only  be  inter¬ 
preted  as  true  EEG  potentials  related  to  verbal  behavior  (overt  or 
covert),  and  not  attributable  to  auditory  or  visually  evoked  potentials 
muscle  or  movement  artifacts,  or  eye  movements  (see  Figures  3,  4,  5  and 
o  the  First  Semi-Annual  Technical  Progress  Report  and  the  discussion 
regarding  artifacts  on  pages  13-19  of  that  report). 
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Nevertheless,  we  cannot  know  from  that  data  what  contribution 
eye  movements  may  have  made  to  the  current  data.  Therefore,  a  control 
study  using  five  words  from  Table  1  was  conducted,  where  eye  movements 
per  se  were  measured  along  with  the  EEG.  Six  types  of  stimulus-response 
combinations  were  obtained  for  each  of  the  five  words,  each  repeated 
five  times.  The  words  were  TIP,  HAD,  PUT,  COUGHDROP,  and  SCHOOLBOY. 

The  conditions  were: 

(1)  Visual  stimulus  presentation,  overt  response,  eyes  open. 

(2)  Visual  stimulus  presentation,  covert  response  (silent  reading), 
eyes  open. 

(3)  Auditory  stimulus  presentation,  overt  response,  eyes  open. 

(4)  Auditory  stimulus  presentation,  covert  response,  eyes  open. 

(5)  Auditory  stimulus  presentation,  overt  response,  eyes  closed. 

(6)  Auditory  stimulus  presentation,  covert  response,  eyes  closed. 

The  S  was  instructed  to  remain  as  relaxed  as  possible  between  and 
during  trials.  On  hearing  a  warning  click  (CLK)  in  her  earphones  for 
eyes  open  conditions,  S  was  to  fixate  on  a  spot  in  the  center  of  the 
stimulus  screen.  A  visual  or  auditory  stimulus  was  presented  1  sec 
later,  followed  in  1  sec  by  another  CLK.  On  hearing  the  second  CLK, 
the  S  was  to  respond  either  overtly  or  covertly  according  to  the  condi¬ 
tion.  Approximately  30  sec  later,  the  next  trial  began,  with  a  rest 
of  2  min  between  conditions.  In  all  trials,  eye  movements  were  de¬ 
tected  by  two  pairs  of  electro-oculograph  (EOG)  electrodes  (one  pair 
for  vertical  movement,  one  pair  for  horizontal)  placed  around  the  eyes. 

The  results  may  be  summarized  as  follows: 

(1)  For  the  25  visual  stimuli,  overt  response  (worse  case  for  eye 
movements)  ,  eye  movements  were  detected  on  both  the  EOG  and 
the  four  EEG  channels  when  S  attended  to  the  fixation  poinc 
on  hearing  the  warning  CLK. 

(2)  Of  these  25  presentations,  nine  (38%)  had  eye  movement  arti¬ 
facts  in  the  EEG  record  as  confirmed  by  the  EOG  electrodes. 

Of  these  nine,  only  four  could  be  associated  with  the  actual 
response  period;  the  remainder  were  randomly  distributed. 

Thus,  only  16%  of  the  eye  movement  artifacts  could  have  con¬ 
tributed  to  a  correct  classification,  ^if  all  of  the  artifacts 
occurred  in  the  same  way  for  a  given  word,  which  they  did  not. 

(3)  For  all  other  conditions,  the  amount  of  eye  movement  artifacts 
decreased  so  that  by  condition  six  (auditory  stimulus,  eyes 
closed,  covert  response),  no  artifacts  were  obtained  at  all. 
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<4>  artifacts  a“d  t TolT'  T"‘"  those  a  falr  amount  of 

i  S  containing  no  artifacts,  a  specific  poten- 

tial  change  in  the  EEC  was  identifiable  in  the  raw  record 
that  could  be  related  to  the  verbal  behavior,  whether  overtlv 

had  the  t ime'cour se^a nd 'waveform ^f 2 1 hos e 

•~~ement  artlfacl;s-’  even  when  the  EOG  electrodes  showed  that 

tongue* i  oAlth°tUgh  artlfaCtS  °f  S°me  -ri  ZZZoT~ 
tongue  movements,  cannot  be  completely  ruled  out,  the  proba- 

signals^  P~  "  ^  ™  ~ 

Figure 1  l^illust  rates  "an 1  (A) 'over  tS°nd  Tb) ToTe  B< 

stimulus  SCHOOLBOY.  under  ™ndltlon_  /  t  resP°nse  to  the  visual 

significance  during  “ThTt”*  ~ 

vertical  and  horizontal  EOG  electrodes  the  channels  are  the 

channels;  the  last  is  the  OK  rnH  .  ’  .  G  t  four  are  the  four  EEG 

are  definite  BOO  potentt!l^  h  ™  channel>  Note  that  there 

following  the  nrsr^LK  ^!na^la  tS  in  th*  EEG  in  Fi^e  «A 
Preceding  the  onset  of  vnr  l ■  Bject  B  attends  to  the  fixation  point. 

positive  shift  is  obtained  i^he^thaf  nlt?  ■  Sl°W_waVe  ^gative- 
This  slow-wave  is  also  seen  during  contlnues  during  vocalization. 

This  potential  is  evidently  the  pat “^7^0  chSP°”7  *"  Fie“re  HB’ 
electrode  F7  anH  „„  ^  change  (more  obvious  in 

covert^'prLLed  Pi””6 Wheth6r  " 

on  the  horizontal  electrodT^l: i*.  'd  g 
zation  and  presu„ed  silent  reading.  Hewer.  ,1™ 

f leant  size  in  either  the  overt  nr  I  since  it  is  not  of  signi- 

during  a  small  portion  of  either  the  FFr°VGr  reSponse-  and  fasts  only 

zation.  it  cannot  co^tr^^ch  either  ZTnT^LT  ^  V°Cali- 
classification.  11  artlfact  or  in  response 

SCHOOLhOY^ut^in  hothrcasesSh°WS  “'C'  ““  °Vert  and  covert  fOSPOnses  to 

fact  as  confirmed  by ^  S’eSSSr'  llTTI  ^  I™ 

(Figure  15A) ,  the  eye  movements  occur  variable  th°  ‘I  the  °Vert  resP°nses 

period,  not  just  in  association  with  the  verbal response* 

are  more  likely  to  produce  confusion  during  classif lent  „  t7  7y 
classification  However  H,,r-i  B  lassit ication  than  aid  in 

and  T5  which  was  seen  in Figur^raidT- °n’  i  ^ ^  resp°nse  of  F7 

Figure  15A  and  to  a  reduced  extent  in  Figure3 

is  not  associated  with  a  particular  EOG  e  0  Sf /MpoMe) '  and 
if  response  classification  were  only  carried J  1  SUgf f S  that 

to^AeM  subgroup0?  lass  if  ier  o^araT  10  tQ  1'5  aec^corres  pending 
in  15A  would  be  correctly  classifieHLJilTSj  ^  reSP°nSe 

In  addition  to  the  above  results  ui-h-vi  m 

following  points  show,  on  logical  grinds  ^  "°vei"e"t  the 

s  >-01  grounds,  that  eye  movement  artifacts 
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COVERT  RESPONSE 


EEG  IN  ABSENCE  OF  EYE  MOVEMENTS,  OVERT  AND  COVERT  VERBAL  RESPONSE  TO  SCHOOLBOY,  SUBJECT  B7 


U)  OVERT  RESPONSE 


TO  SCHOOLBOY.  SUBJECT  B7 


did  not  contribute  significantly  to  correct  computer  classifications  in 
this  project  (although  they  may  well  have  contributed  to  confusion), 

(1)  Figure  16  shows  the  EEG  templates  (electrode  F7)  for  Subject  C, 
Session  5,  channel  3,  for  the  words  TIP  and  COOL.  It  also 
shows,  between  the  templates,  three  individual  EEG  TIP  res¬ 
ponses  on  channel  3  that  were  confused  with  COOL.  It  appears 
that  all  three--TIP  2,  TIP  3,  and  TIP  4 — were  probably  confused 
with  COOL  because  of  a  time  shift  in  their  responses  to  the 
right  (see  "cursor")  with  respect  to  the  TIP  template  that 
made  their  slow-wave  responses  more  coincident  with  the  slow- 
wave  of  COOL  than  with  TIP. 

(2)  Eye  movement  artifacts,  when  they  occur,  produce  a  fair  amount 
of  variability  during  a  given  response  period  (Figure  15A) 
and  between  response  periods.  Therefore,  this  variability 
should  show  up  on  the  averaged  records.  Yet,  in  Figure  11, 
the  variability  of  the  EEG  is  less  than  with  EMG,  a  result 
that  is  inconsistent  with  the  idea  of  eye  movement  artifacts 
contaminating  the  record. 

(3)  Eye  movement  artifacts,  when  they  occur,  take  place  when  the 
light  stimulus  first  goes  on  and  the  S  attends  to  the  stimulus 
screen  as  in  Figure  14A  (over  1  sec  before  the  actual  response 
of  the  S) .  Therefore,  if  eye  movement  artifacts  are  present 
in  our  templates  in  a  consistent  way,  then  the  artifacts 
occurring  at  the  onset  of  stimulation  should  be  present  in  the 
averaged  records;  yet  nothing  of  this  sort  is  seen  in  the 
majority  of  templates  (e.g.,  Figures  10  and  11). 

(4)  If  eye  movement  artifacts  are  contributing  significantly  to 
correct  classification  of  the  EEG  response,  then  a  greater 
consistency  would  be  expected  between  EEG  channels  in  their 
percentage  of  correct  classification  than  was  fcund.  That 
is,  we  should  expect  the  eye  movement  artifacts  to  be  con¬ 
sistent  on  a  given  channel  if  they  contribute  to  computer 
classification.  In  this  case,  all  EEG  channels  should  classify 
equally  well.  However,  this  was  not  true;  some  EEG  channels 
proved  to  be  better  classifiers  than  others  (though  not  in 

any  consistent  way  across,  between,  or  within  Ss) . 

In  any  event,  in  Group  III  experiments  (see  Discussion,  below),  we 
will  attempt  to  increase  response  classification  perfox’mance  by  removing 
as  many  of  these  errors  as  possible,  and  to  assess  the  contributions  to 
success  or  failure  of  those  that  cannot  be  removed. 


Classification  by  Other  Statistics 


In  addition  to  the  average  template  comparisons  described  above, 
several  other  statistics  were  also  calculated  for  Subject  C,  Session 
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TIP  AVE 


VOCALIZATION 

FIGURE  16  THREE  EEG  RESPONSES  (ELECTRODE  F7,  SUBJECT  C5)  TO  TIP 
CONFUSED  WITH  COOL  DUE  TO  TIME  SHIFT 
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including  linear  coherence  (Coh) ,  weighted-average  coherence  (Ave  CoM 
and  spectral  densities  (Freq)  for  auto-  and  cross-spectra  (seethe  ’ 

ppendix  for  rationale  and  equations).  The  results  were  uniformly 
disappointing  as  follows:  y 


Correct 

Responses  per  150 

Statistic 

Channels 

Coh 

Ave  Coh 

Freq 

Both  EMC 

31 

14 

34 

All  EEC 

17 

10 

8 

All  Electrodes 

25 

17 

16 

Because  of  these  results  on  Subject  C, 
lated  for  the  other  two  Ss. 


these  statistics  were  not  calcu- 


Since  all  three  statistics  are  ba  ed  on  frequency  analysis 


One  finding  in  the  spectral  density  plots,  at  least  for  Subiect  C 

spherea(EEGr?6)1SanaCtitityn°CCUrred  th®  nondominant  right  hemi-  ’ 

P  (EEC  T6)  than  in  the  homologous  electrode  (EEC  T5)  over  the 

dominant  (speech)  hemisphere.  Since,  in  this  S,  T6  (nondoml'anU 

beTEEOSJaIs  Jhe  P°°^St  ™C  ClaSsl,iers  a"d  T5  responses  were  the 

variability  in  theT  t°''  ^  5  alpha  dulerGIICG  may  be  related  to  the 
will  ho  f  ^  ^  tW°  ternplates  for  response  classification.  This 

fUrth6r  exan,ined  d^ing  collection  and  analysis  of  Group  III  data. 
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DISCUSSION  AND  CONCLUSIONS 


Significance  of  Results 


(1)  For  all  three  Ss  tested,  a  high  percentage  of  individual  word 
utterances  were  correctly  classified,  based  only  on  machine  recognition 
of  biological  signals.  There  was  some  variability  in  this  capacity. 

Within  each  S,  the  second  session  had  a  higher  rate  of  correct  classi¬ 
fications  than  the  first,  indicating  a  learning  or  habituation  effect 
that  probably  reduced  the  response  variability.  There  were  also  differ¬ 
ences  among  Ss,  due  to  differences  in  the  variability  among  them,  with 
the  responses  of  Subject  C  classifying  more  correctly  than  either  B  or 
D.  Nevertheless,  for  all  three  Ss ,  variability  in  their  individual 
templates  was  relatively  low,  and  the  rate  of  correct  classifications 
was  high — generally  over  50%.  The  numbers  correctly  classified  were 
significant  with  a  probability  well  below  .001  that  this  occurred  by 
chance. 

(2)  The  greatest  variability  within  Ss  was  between  types  of 
biological  recordings.  EMG  responses  were  almost  twice  as  good  as 
classifiers  as  were  FEG  responses,  although  the  two  taken  together  were 
better  than  either  group  (or  individual  electrodes  within  a  group)  alone. 

This  finding  suggests  that  more  than  one  electrode  should  be  used  in  a 
biocybernetic  communication  system.  It  is  supported  by  the  result  that 
there  was  no  evidence  in  any  of  the  Ss  that  a  given  electrode  of  either 
the  EMG  or  EEG  groups  was  consistently  better  as  a  classifier  than  any 
other  electrode  in  the  group.  One  exception  to  this,  however,  is  that 
in  homologous  EEG  electrodes  T5  and  T6,  in  Subjects  C  and  D,  T5  was  a 
predominantly  better  classifier  than  was  T6,  while  in  Subject  B  the 
opposite  was  true.  This  finding  is  possibly  related  to  the  results 
from  the  spectral  analysis  in  Subject  C,  that  T6  contained  consistently 
higher  amplitudes  of  alpha  frequencies  than  T5, 

(3)  Another  source  of  variability,  shown  by  the  F-ratios,  was  the 
part  of  the  biological  response  analyzed.  It  is  evident  that  if  only 
that  portion  of  the  biological  response  associated  with  the  actual  verbal 
response  is  used,  then  the  rate  of  correct  classification  increases. 

This  may  actually  be  easier  to  do  with  continuous  speech  (or  thought) 
than  single  words,  since  variability  with  single  words  is  greatest  between 
words. 

(4)  As  expected  from  Group  I  results,  each  S's  biological  responses 
associated  with  speech  are  unique  to  him,  as  confirmed  by  the  Between 
Subjects  results.  It  is  of  scientific  interest  that  a  fair  number  of 
EMG  responses  were  significantly  classified  correctly  between  Ss,  indi¬ 
cating  that  muscle  activity  for  verbal  expression  may  be  more  invariant 
between  individuals  than  has  been  supposed  (MacNeilage  and  MacNeilage,  1971). 


I 

3 

3 

This  result  also  means,  however,  that  unique  templates  must  be  built  for 
each  individual  using  a  biocybernetic  communication  system. 

(5)  A  surprising  result  from  the  confusion  matrices  was  the  finding 
that  emphasis  on  different  parts  of  the  bisyllabic  words  did  not  mater¬ 
ially  effect  biological  responses.  This  should  reduce  possible  future 
problems,  since  the  library  of  templates  in  a  biocybernetic  communication 
system  may  be  reduced  to  tolerable  dimensions  if  emphasis  on  all  accented 
words  can  be  ignored. 

(6)  One  additional  source  of  error  identified  in  our  results  may 
be  eliminated  by  additional  computer  processing,  while  two  others  may 
not  be  eliminated  at  all.  The  first  is  the  time  shift  in  a  biological 
response  with  respect  to  the  onset  of  vocalization  (or  verbal  thinking) 
from  one  response  to  another.  We  do  not  know  how  many  errors  occurred 
due  to  this  shifting,  but  the  error  can  be  removed  by  time  justification 
using  a  minimum  difference  criteria  between  two  responses  before  com¬ 
bining  them  to  make  a  template,  and  before  calculating  an  RMS  difference 
for  classification.  This  will  be  done  on  Group  III  data. 

(7)  The  other  two  errors  we  may  have  to  accept — namely,  amplitude 
and  movement  artifacts.  The  first  of  these  may  possibly  be  corrected 
for  by  an  automatic  gain-change  feature  in  biocybernetic  amplifiers,  or 
by  identification  through  context,  or  both.  The  second  of  these,  and 
especially  eye  movement  artifacts,  may  be  reduced  when  speech  (or 
thought)  is  continuous,  by  time  shifting  or  by  closing  the  eyes.  We 
are  confident  from  the  results  of  our  control  study  that  eye  movements 
did  not  overly  contaminate  our  records  to  reduce  classification  rate,  and 
certainly  were  not  the  major  EEG  parameter  that  contributed  to  correct 
classification  (despite  the  fact  that  parts  of  some  of  the  EEG  averages 
look  like  artifact). 

To  conclude,  we  believe  that  the  results  of  thi3  year's  research, 
and  particularly  the  results  reported  herein,  have  demonstrated  beyond 
our  expectations  that  a  biocybernetic  communication  system  using  biolo¬ 
gical  information  related  to  speech  is  definitely  feasible.  During  the 
next  year,  we  expect  to  find  out  if  it  is  also  feasible  for  verbal 
thinking. 


Future  Research — Group  III  Data 


Details  of  the  work  to  be  conducted  in  the  second  contract  year  are 
in  SRI  Proposal  No.  LSU  72-133,  dated  October  30,  1972.  In  the  light  of 
the  present  results,  it  would  be  useful  to  list  the  specific  objectives. 

(1)  Biologic  response  measurements  will  be  made  over  shorter 

intervals  of  time  to  correspond  with  verbal  (overt  and  covert) 
responses , 
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(2)  Instrumentation  will  be  used  to  monitor  all  types  of  artifacts. 

Samples  containing  artifacts  of  any  sort  will  be  divided  into 
subgroups  according  to  type,  and  classified  separately  to 
assess  the  contribution  of  artifact-rich  relative  to  artifact- 
poor  measures. 

(3)  All  other  sources  of  error  described  above  will  be  eliminated 
as  nearly  as  possible. 

(4)  Attempts  will  be  made  to  identify  the  smallest  unit  of  speech 
that  can  be  correctly  classified  with  a  minimum  template 
vocabulary  or  library. 

(5)  An  effort  will  be  made  to  assess  the  contributions  of  the  EEG 
from  dominant  and  nondominant  speech  areas  of  the  brain  to 
correct  classification. 

(6)  An  attempt  will  be  made  to  identify  the  various  components  of 
a  biological  response  related  to  overt  and  covert  speech, 
including  mechanical,  semantic,  contextual,  and  affective 
components. 

(7)  Finally,  and  most  important,  we  will  primarily  attempt  to 
classify  covert  verbal  behavior,  or  thinking,  with  biological 
signals  alone. 
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Appendix 


DISCUSSION  OF  STATISTICS  USED  FOR  MACHINE  RECOGNITION 


To  classily  the  words  in  the  test  vocabulary  automatically,  it  is 
necessary  to  have  both  a  procedure  o!  combining  the  data  from  the  mea¬ 
surements  and  a  statistic  whose  numerical  value  determines  the  final 
classification.  Several  statistics  have  been  tried  in  this  project, 
and  the  formal  mathematical  definitions  for  each  are  given  below. 


Introduction  and  Terminology 


tte  believe  that  it  is  helpful  to  clarify  some  basic  facts  about 
the  recognition  process  in  general,  before  describing  our  particular 
recognition  application  using  bioelectric  potentials.  The  purpose  of 
all  our  statistics  is  to  enable  accurate  recognition  so  that  the  signals 
can  be  used  lor  biocybernetic  communication  purposes.  The  two  main 
approaches  used  are  template  matching  based,  first,  on  the  pattern  of 
the  signal  over  time,  and  second,  on  the  pattern  over  frequency  of  the 
power  spectral  density  function  of  the  signal.  Recognition  allows  us 
great  economy  in  biocybernetic  communication  because  we  may  substitute 
the  label  or  recognition  category  for  the  signal  itself. 


Recognition  means  the  labeling  of  some  measurement  data  according 
to  pre-established  criteria  or  boundaries.  In  our  case,  we  use  the  names 
of  the  words  themselves— e.g.  ,  HAD,  PUT,  and  so  on— as  the  labels  for 
groups  of  measurements  that  were  taken  when  the  subject  was  saying  these 
words.  Recognition  requires  such  labels  to  be  supplied,  whereas  cluster¬ 
ing  does  not.  "Clustering"  means  the  partitioning  of  all  the  measure¬ 
ments  into  sel f -consistent  or  homogeneous  groups,  without  reference  to 
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any  preconceived  labels.  Clustering  handles  the  measurements  as  they 
are,  whereas  recognition  fits  the  measurements  to  external  labels.  A 
clustering  method  finds  an  appropriate  number  of  valid  groups  so  that 
the  clustering  is  an  accurate  but  abbreviated  description  (summary  or 
template)  of  the  data.  Clustering  facilitates  recognition  usually  by 
providing  a  rational  objective  method  of  labeling  the  measurements.  The 
recognition  labels  may  be  rather  arbitrary  and  not  logically  related  to 
the  measurements,  except  by  convention,  established  practice,  or  pref¬ 
erence  . 

Spectral  Analysis"  is  a  method  of  decomposing  any  time  waveform 
into  a  set  of  sinusoidal  waves  that  can  be  combined  to  reconstruct  the 
original  waveform.  There  are  two  kinds  of  spectral  analysis:  the  usual 
kind,  which  wo  have  used,  which  ignores  the  phase  of  the  frequency  com¬ 
ponents,  and  a  more  .-sophisticated  kind,  called  complex  spectral  analysis, 
which  gives  phase  information  as  well.  We  will  discuss  only  the  first- 
kind  here.  In  future  work,  we  will  apply  a  spectral  analysis  program 
that  will  take  phase  into  account  (Singleton,  1967).  The  purpose  of 
spectral  analysis  is  to  transform  the  waveform  or  time  description  of  a 
signal  into  a  frequency  description  in  the  hope  that  recognition  of  the 
signal  can  bo  made,  assuming  the  frequency  description  corresponds  more 
closely  to  the  recognition  requirements  than  does  the  time  description. 

Clustering  Approach  to  Recognition 

It  had  been  our  original  intention  to  apply  the  clustering  program 
ISODATA  to  the  data.  The  computation  time  for  one  iteration  through 
the  data  for  this  program  is  given  approximately  by  the  following  formula: 

T  =  NCOLS  V  NROWS  v  NPATS  X  M  seconds 
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mm*: 


where 


NCOLS  =  the  number  of  variables  (columns) 

NROWS  =  the  number  of  clusters  (rows) 

NPATS  =  the  number  of  patterns 

M  =  a  factor  depending  on  the  basic  cycle-time  of  the 
computer. 

In  the  case  of  Group  II  data,  there  are  15  words,  so  we  can  hope¬ 
fully  find  15  clusters — one  cluster  for  each  word.  By  evaluating  NCOLS, 
NROWS,  and  NPATS  and  estimating  a  value  for  M,  it  was  found  that  the 
computation  time  and  the  computer  memory  size  required  were  rather  large 
for  this  project.  Thus,  to  save  computation  time  necessary  for  the 
iterative  searching  procedure  of  ISODATA,  and  because  we  would  have  had 
to  modify  the  program  to  expand  its  memory  requirements  to  accommodate 
the  large  number  of  data  samples,  we  economized  on  computation  expenses 
by  writing  a  noniterative,  special-purpose  "clustering"  program  for  this 
particular  application.  Because  the  data  are  sufficiently  well-clustered, 
this  approach  is  satisfactory  at  the  15-cluster  level,  so  the  usual  itera¬ 
tive  search  procedure  was  not  necessary,  and  templates  formed  from  aver¬ 
ages  of  the  data  in  each  category  were  used  as  the  cluster  cente- . 

Let  X  be  a  basic  data  value  for 
st  l.jkf, 

subject  s,  s  =  B,  C,  or  D 
session  t,  t  =  1  or  2 
time  i,  1  <  i  <  255 
channel  j,  1  <  j  <  6 
word  category  k,  1  <  k  <  15 
repetition  i,  1  <  SL  <  10 

The  data  were  digitized  to  9  bits  accuracy,  so  -255  <  x  <  255 

st  i jkj& 

for  all  i,  j,  k,  i,  f5,  and  t  as  above. 
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The  template  or  average  value  is  computed  by  averaging  (Clynes  et  al., 
1987)  each  point  in  the  ten  responses  (i.e.,  repetitions)  within  a  known  ” 
uoicl  category  (k)  .  Thus,  for  any  word  category  k, 


A 

st  ijk 


10 

X 

10  st'  i Jkj& 

1=1 


The  HUS  di  I  Terence  t^)  between  the  ilh  utterance  of  word  k  and  the 
template  for  word  category  .  (1  <  m  S  15),  where  c  denotes  the  combine- 
tion  of  time  samples  and  channels,  is  given  by: 


c  k/m  /  N 


ltc  jec 


lJM  s2t2  ijm 


where  is  the  number  of  data  points  in  combination  c  and 


S,t9AiJm  lf  si  *  s2  °r  4i  /  l2  »>  *  k 


S2V iJm 


10  A  -  x 
sjll  iJk  s  t  ijkA 


11  S1  =  S2  and  li  t2  and  >n  =  k 


h -Ratio  Test  for  Significance  of  Variance 
Between  and  Within  Templates 

An  I- -statistic  was  calculated  for  each  point  in  time  for  each  channel 
and  for  each  session,  and  a  plot  was  made  of  each  F-value  versus  time. 
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Let  F  .  be  a  F  value  at 
st  ij 

time  i,  1  <  i  <  255 

channel  j ,  1  <  j  <  6 

subject  s ,  s  =  B,  C,  or  D 

session  t,  t  =  1  or  2 

then 


SSB  ./(15  -  1) 

F  =  -  st  jJ _ 

st  ij  SSW.  /[ 15(10  -  1)1 

st  ij 


where 


stSSBij  =  sum  of  squares  between  templates 
stSS"'ij  =  SUm  °f  squares  within  templates 
15  -  1  =  14  -  degrees  of  freedom  in  numerator 
15(10  1)  =  135  =  degrees  of  freedom  in  denominator. 


|  15  /  io  \  r  15  /  10 

lk=i  \*=i  /  Lk=i  \  n~i 


15  r/  io 


vIE 


k=l  LU=1 


x 

st  ijkjJ 


-uz 


X 

st  ijia 


where  stxij.k^  is  the  same  as  previously  defined. 

From  a  table  of  the  F-statistic  (Selby,  1971),  values  greater  than 
1.77  are  significant  at  the  .05  level,  and  values  above  2.23  are  signif 
cant  at  the  .01  level.  In  most  cases,  the  F-statistic  showed  that  the 
time  segment  over  which  the  classification  was  significant  was  from  the 
middle  subgroup  samples  101-200. 


'.v  -..ip  tit." 


Combinations  of  Data  Points  and  Channels 

Based  on  the  results  with  the  F-ratio  test,  various  combinations 
of  points  in  time  and  various  channels  (electrode  placements)  were  used 
to  explore  time  segments  and  electrode  placements  to  determine  which 
combinations  would  give  the  highest  correct  identification  of  the  words. 
The  samples  in  time  were  divided  into  subgroups — i.e.,  samples  1-100 
being  the  first  subgroup,  samples  101-200  the  second  subgroup  (where 
the  F-ratios  were  significant),  and  201-255  the  final  subgroup.  In  addi¬ 
tion,  all  samples  (1-255)  were  used,  giving  four  groups  of  time  points. 

All  six  channels  were  treated  individually.  In  addition,  EMG  chan¬ 
nels  (1  and  2)  were  grouped  together,  EEG  channels  (3,  4,  5,  and  6)  were 
giouped  together,  and  finally  all  channels  were  grouped  together,  giving 
nine  different  channel  combinations. 

Thus,  with  the  four  time  segments  there  were  36  different  channel¬ 
time  combinations.  Typical  values  of  N  (used  in  calculating  the  RMS 
difference)  are: 


Combination 


Channel  1,  samples  101-200  100 
EMG  (Channels  1  and  2),  all  samples  510 
All  channels,  all  samples  1530 


Closest-Match  Recognition  and  Confusion  Matrix 


The  recognition  or  classification  of  each  test  word  was  accomplished 
by  matching  it  to  the  set  of  all  templates.  The  best  match  is  word  w 
where 

m=15 


R,  „  =  min 

c  k£w 

m=l 


Ic'W  cRk.!2-cV-cVl5  °CCUrS* 
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If  vv  =  k  for  the  minimum  value,  this  is  considered  a  correct  match; 
if  w  /  k,  an  incorrect  match  was  found. 

A  confusion  matrix  (C  )  was  computed  as  the  table  for  the  number 

kw 

of  times  word  k  was  classified  as  being  word  w.  Matrices  were  formed 
for  each  combination  of  time  samples  and  channels.  The  main  diagonal 
(where  k  =  w)  gives  a  tally  of  the  number  of  correct  matches  for  each 
word.  A  chi-square  statistic  was  calculated  based  on  these  correct 
matches  to  test  for  significance  (nonrandomness). 

The  Chi-Square  Test  of  Significance 

As  there  are  15  different  words,  ihe  probability  of  picking  a  par¬ 
ticular  word  correctly  by  chance  is  1/15.  There  are  ten  repetitions  of 
each  word,  giving  10/15  as  the  expected  number  of  correct  responses 
assuming  random  data. 

Two  chi-square  statistics  were  computed:  (1)  a  chi-square  with  15 
degrees  of  freedom,  which  tests  significance  of  recognition  of  the  15 
words;  and  (2)  a  chi-square  with  1  degree  of  freedom,  which  tests  the 
significance  of  the  total  number  of  responses  correctly  classified, 
summed  over  all  words. 

Let  C  be  the  general  element  of  the  confusion  matrix,  as  defined 
kw 

earlier — i.e.,  C  is  the  number  of  times  word  k  gets  classified  as  if 
kw  2 

it  were  word  w.  Let  \  be  the  chi-square  statistic  with  1  degree  of 

freedom,  and  X  be  the  chi-square  statistic  with  15  degrees  of  freedom. 
1 5 

Then,  for  one  session, 
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Critical  values  for  rejection  ol  the  hypothesis  ol‘  no  significant 
dillerence  between  the  observed  sample  and  a  random  population  were  6.64, 
P  -  *01.  l'or  1  degree  of  freedom,  and  30,58,  p  i"  .01,  for  15  degrees  of 
freedom. 


Spectral  Analysis  Using  the  Fast  Fourier  Transform 

The  method  of  transforming  a  waveform  in  the  time  domain  to  a 
mathematically  equivalent  representation  in  the  frequency  domain  is 
known  as  a  "Fourier  transformation."  The  computational  implementation 
of  this  transform  on  a  computer  is  called  a  "fast"  transform,  because  a 
particular  algorithm  is  required  to  make  such  computations  efficient. 

For  this  project,  we  used  a  computer  program  supplied  to  us  by  one  of 
our  project  consultants,  Dr.  Gary  Galbraith.  This  program  came  from  the 
Biomedical  Data  (BMD)  series  developed  at  UCLA,  was  adapted  by  Galbraith 
for  his  computer,  and  then  readapted  by  Wolf  for  the  SRI  CDC-G400. 
Galbraith  first  used  a  spectral  analysis  program  that  is  documented 
(Dixon,  1970)  as  BMD02T  "Autocovariance  and  Power  Spectral  Analysis." 
After  gaining  experience  with  this  particular  program,  Galbraith  updated 
his  approach  to  the  newer  version  in  the  same  series,  which  is  cataloged 
as  91  v  FF. 
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N’ame  Lines  of  FORTRAN  Purpose 

MAIN  PROGRAM.  READS  TAPE,  APPLIES  WINDOW 
COMPUTES  SPECTRA 
PRINTS  VALUES 

COMPUTES  AVERAGE  COHERENCE 

The  output  of  the  program  first  lists  the  description  of  the  data 
on  the  tape  such  as  the  subject's  number  and  date  of  recording.  Next, 
the  coherence  (as  defined  below)  between  all  possible  pairs  of  channels 
is  given  lor  each  ol  the  33  bins  of  the  frequency  spectrum  we  employed 
(0  to  19  Hz).  Since  we  used  six  channels,  there  are^j  15  different 
combinations  of  channels  taken  two  at  a  time  for  cross-spectral  ampli¬ 
tudes,  and  six  auto-spectral  amplitudes,  making  a  total  of  21.  Similarly 
tor  the  quantity  C',  there  are  another  21  values  for  each  of  the  bins. 

The  Coherence  and  Weighted-Average  Coherence  Statistics 

Linear  coherence  is  a  statistic  that  provides  an  estimate  of  the 
degree  to  which  a  particular  frequency  component  in  the  EEG  of  one  brain 
region  is  related  to  an  identical  frequency  component  in  the  EEG  of  an¬ 
other  brain  region.  Thus,  in  a  cross-seetral  analysis  as  many  coherence 
values  may  be  calculated  as  there  arc  frequency  bins  (Walter,  1965). 

The  weighted-average  coherence."  on  the  other  hand,  is  a  statistic  de¬ 
rived  from  cross-spectral  frequency  analysis  parameters  that,  in  a  sense, 
summarize  over  the  several  linear  coherences  in  a  cross-spectrum  to  give 
a  single  number  that  measures  the  overall  degree  of  interaction  between 
two  brain  regions  (Galbraith,  1967).  The  term  "weighted”  refers  to  the 
fact  that  the  EEG  ol  the  two  regions  used  in  computation  must  be  large 
in  amplitude  and  highly  coherent. 

following  Galbraith  (1966,  1967)  we  define  the  coherence  function, 
at  frequency  f,  as 
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where 

A  is  the  cross-spec t  ml  density 
xy 

A  is  the  autospec t ml  density  of  Channel  x 
x 

A^  is  the  autospeetrnl  density  of  Channel  y. 

C ( I' )  is  normalized  and  bounded  between  0.0  (Complete  lack  ol  relationship) 
and  1.0  (perfect  linear  relationship). 

The  weighted  average  coherence  function  is  defined  by 

c  V  A  (I  )  C(1  )]/  7  A  (I  ) 

Z-V  .xy  i  i  i—i  xy  i 

i  i 

w lie  re 

A  ( 1'  )  is  the  cross-spectral  density  at  frequencies  1^ 
satisfying  a  threshold  criterion  that  they  must 
exceed . 

C(l^)  is  the  coherence  at  I requency  f  satisfying  the 
threshold  criterion. 

The  computer  program  prints  out  these  quantities,  which  we  later 
used  as  input  to  the  template  matching  program,  in  place  of  the  original 
time  series  data.  The  use  of  these  spectral  data  or  coherence  statistics 
did  not  provide  as  good  recognition  results  as  obtained  with  the  original 
waveform  data. 
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